Maximum a Posteriori Estimation by Search in Probabilistic Programs

We introduce an approximate search algorithm for fast maximum a posteriori probability estimation in probabilistic programs, which we call Bayesian ascent Monte Carlo (BaMC). Probabilistic programs represent probabilistic models with varying number of mutually dependent finite, countable, and continuous random variables. BaMC is an anytime MAP search algorithm applicable to any combination of random variables and dependencies. We compare BaMC to other MAP estimation algorithms and show that BaMC is faster and more robust on a range of probabilistic models. Introduction Many Artificial Intelligence problems, such as approximate planning in MDP and POMDP, probabilistic abductive reasoning (Raghavan 2011), or utility-based recommendation (Shani and Gunawardana 2009), can be formulated as MAP estimation problems. The framework of probabilistic inference (Pearl 1988) proposes solutions to a wide range of Artificial Intelligence problems by representing them as probabilistic models. Efficient domain-independent algorithms are available for several classes of representations, in particular for graphical models (Lauritzen 1996), where inference can be performed either exactly and approximately. However, graphical models typically require that the full graph of the model to be represented explicitly, and are not powerful enough for problems where the state space is exponential in the problem size, such as the generative models common in planning (Szörényi, Kedenburg, and Munos 2014). Probabilistic programs (Goodman et al. 2008; Wood, van de Meent, and Mansinghka 2014) can represent arbitrary probabilistic models. In addition to expressive power, probabilistic programming separates modeling and inference, allowing the problem to be specified in a simple language which does not assume any particular inference technique. Recent success in PMCMC methods enables efficient sampling from posterior distributions with few restrictions on the structure of the models (Wood, van de Meent, and Mansinghka 2014; Paige et al. 2014). However, an efficient sampling scheme for finding a MAP estimate would be different from the scheme for inferring Copyright c 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. the posterior distribution: only a single instantiation of model’s variables, rather than their joint distribution, must be found. This difference reminds of the difference between simple and cumulative reward optimization in many settings, for example, in Multi-armed bandits (Stoltz, Bubeck, and Munos 2011): when all samples contribute to the total reward, the algorithms are said to optimize the cumulative reward, which is the classical Multi-armed bandit settings. Alternatively, when only the quality of the final choice matters, the algorithms are said to optimize the simple reward. This setting is often called a search problem. Previous research demonstrated that different sampling schemes work better for either cumulative or simple reward, and algorithms which are optimal in one setting can be suboptimal in the other (Hay et al. 2012). In this paper, we introduce a sampling-based search algorithm for fast MAP estimation in probabilistic programs, Bayesian ascent Monte Carlo (BaMC), which can be used with any combination of finite, countable and continuous random variables and any dependency structure. We empirically compare BaMC to other feasible MAP estimation algorithms, showing that BaMC is faster and more robust.